Methods and devices to identify focal objects

ABSTRACT

A method includes dividing a field of view into a plurality of zones and sampling the field of view to generate a photon count for each zone of the plurality of zones, identifying a focal sector of the field of view and analyzing each zone to select a final focal object from a first prospective focal object and a second prospective focal object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/122,667, filed on Dec. 15, 2020, which application is herebyincorporated by reference herein in its entirety.

TECHNICAL FIELD

This application relates to methods and devices for identifying focalobjects and distances in a field of view.

BACKGROUND

Time of Flight systems may be used to detect objects and theirrespective distances from the lens of a camera to provide the cameradata needed to focus the camera. This may be referred to asautofocusing. However, there are many challenges associated withselecting the appropriate object or distance for autofocusing.Autofocusing may also cause flicker if the focal object distance issubject to abrupt changes.

SUMMARY

In accordance, with an embodiment of the present application, a methodto identify a focal object in a field of view, the method may include:dividing a field of view into a plurality of zones; identifying a focalsector occupying a subset of the plurality of zones; analyzing each zoneof the plurality of zones within the focal sector to identify a firstprospective focal object and to identify a second prospective focalobject; determining a size of the first prospective focal object in thefield of view and determining a size of the second prospective focalobject in the field of view; selecting a final focal object from thefirst prospective focal object and the second prospective focal objectby comparing the size of the first prospective focal object with thesize of the second prospective focal object; and focusing a lens of acamera depending on the final focal object.

In accordance, with an embodiment, a Time of Flight system configured todetect a photon count for each zone of a plurality of zones of a fieldof view includes: a processor in communication with the Time of Flightsystem and a memory comprising an instruction set to be executed in theprocessor, the instruction set when executed causing the processor to:identify a focal sector occupying a subset of the plurality of zones,analyze each zone of the plurality of zones within the focal sector toidentify a first prospective focal object and to identify a secondprospective focal object, determine a size of the first prospectivefocal object and determine a size of the second prospective focalobject, and select a final focal object from the first prospective focalobject and the second prospective focal object by comparing the size ofthe first prospective focal object with the size of the secondprospective focal object, the final focal object being located at afinal-focal-object distance from the Time of Flight system. The Time ofFlight system may further include a lens focus system configured tofocus a lens of a camera depending on the final focal object.

Consistent with an embodiment, a method to identify a focal object in afield of view, the method includes dividing a field of view of a Time ofFlight system into a plurality of zones; sampling the field of view withthe Time of Flight system to generate a photon count for each zone ofthe plurality of zones; identifying a focal sector occupying a subset ofthe plurality of zones; analyzing each zone of the plurality of zoneswithin the focal sector to identify a first prospective focal object andidentify a second prospective focal object; determining a size of thefirst prospective focal object by analyzing a photon count for each zoneof the plurality of zones to count each zone of the plurality of zonesthat captures an object within a threshold distance of the firstprospective focal object; determining a size of the second prospectivefocal object by analyzing the photon count for each zone of theplurality of zones to count each zone of the plurality of zones thatcaptures an object within the threshold distance of the secondprospective focal object; selecting a final focal object from the firstprospective focal object and the second prospective focal object bycomparing the size of the first prospective focal object with the sizeof the second prospective focal object, the final focal object beinglocated at a final-focal-object distance from the Time of Flight system;and focusing a lens of a camera depending on the final-focal-objectdistance.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments will now be described, by way of example only,with reference to the annexed figures, wherein:

FIG. 1 illustrates a ToF system 100 of an embodiment;

FIG. 2A shows an enlarged view of an optical source, in accordance withan embodiment;

FIG. 2B shows an enlarged view of an optical receiver, in accordancewith an embodiment;

FIG. 3 depicts at of a histogram;

FIG. 4 depicts a ToF system coupled with a camera in accordance with anembodiment;

FIG. 5 depicts a field of view divided in zones in accordance with anembodiment;

FIG. 6 depicts a Field of view with multiple objects identified in afocal sector in accordance with an embodiment;

FIG. 7 . depicts zones of a field of view where a first prospectivefocal object and a second prospective focal object are present;

FIG. 8 depicts a flowchart of a method for outputting a distance withtemporal filtering in accordance with an embodiment.

FIG. 9 depicts a method to identify a focal object in a field of view inaccordance with an embodiment; and

FIGS. 10A and 10B depict a method to identify a focal object in a fieldof view in accordance with an embodiment.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

In the ensuing description, one or more specific details areillustrated, aimed at providing an in-depth understanding of examples ofembodiments of this description. The embodiments may be obtained withoutone or more of the specific details, or with other methods, components,materials, etc. In other cases, known structures, materials, oroperations are not illustrated or described in detail so that certainaspects of embodiments will not be obscured.

Reference to “an embodiment” or “one embodiment” in the framework of thepresent description is intended to indicate that a particularconfiguration, structure, or characteristic described in relation to theembodiment is comprised in at least one embodiment. Hence, phrases suchas “in an embodiment” or “in one embodiment” that may be present in oneor more points of the present description do not necessarily refer toone and the same embodiment. Moreover, particular conformations,structures, or characteristics may be combined in any adequate way inone or more embodiments.

The references used herein are provided merely for convenience and hencedo not define the extent of protection or the scope of the embodiments.

Time of Flight (ToF) systems are used in many applications to detectobjects in a three dimensional space. In various embodiments, a Time ofFlight system may cast light onto a scene and detect the time it takesfor light to be reflected from objects in the scene back to the Time ofFlight system. The time, along with the speed of light, can be used tocalculate the distances between objects in a three dimensionalenvironment and the ToF systems. This information can be used in manydifferent ways including selecting an object for an autofocusapplication for a camera.

FIG. 1 illustrates a ToF system 100 of an embodiment of the presentdisclosure.

An object 101 is disposed in a three dimensional environment positionedin front of the ToF system 100. The ToF system 100 may be used todetermine the proximity of the object 101 to the ToF system 100. Theobject 101 is provided for explanatory purposes. The three-dimensionalenvironment may include additional objects of various shapes or sizesdisposed at varying distances from the ToF system 100 and the ToF system100 may determine the proximity of the various objects in thethree-dimensional environment. The object 101 may comprise multiplesurfaces at various distances from the ToF system 100, and the ToFsystem 100 may determine the depth of the different surfaces of theobject 101. The ToF system 100 may simultaneously detect the proximityof additional objects in the three dimensional environment from the ToFsystem 100. In various embodiments, the ToF system 100 may also comprisea memory 132. The processor 126 may store data in the memory 132 andretrieve data from the memory 132. The memory 132 may be anon-transitory computer readable medium. The memory 132 may also storeprograms that may be executed by the processor 126. The programs maycomprise instruction sets executed by the processor 126.

The ToF system 100 may comprise an optical source 102 and an opticalreceiver 104.

FIG. 2A shows an enlarged view of the optical source 102, in accordancewith an embodiment.

As depicted in FIG. 2A, the optical source 102 may comprise a pluralityof optical emitters 102-1 to 102-NN arranged as an array. Although theexample of FIG. 2A illustrates the optical emitters 102-1 to 102-NN asbeing arranged in a square N×N array, other array shapes (e.g.ellipsoidal arrays or circular-shaped arrays) may be possible in otherembodiments. Each of the optical emitters 102-1 to 102-NN may compriseone or more infrared sources, modulated light emitting diodes (LEDs), orsemiconductor lasers, or combinations thereof, although other types ofoptical sources may be possible.

In various embodiments, where the optical emitters 102-1 to 102-NNcomprise semiconductor lasers, an optical emitter 102-i of the array ofoptical emitters 102-1 to 102-NN may comprise one or morevertical-cavity surface-emitting lasers (VCSELs), quantum well lasers,quantum cascade lasers, interband cascade lasers, or verticalexternal-cavity surface-emitting lasers (VECSELs), or the like.

The optical emitters 102-1 to 102-NN may be configured to operate at thesame wavelength. In other embodiments, however, the optical emitters102-1 to 102-NN may operate at different wavelengths. For example, thegroup 108 of optical sources and the group 110 of optical emitters 102-1to 102-NN may operate at different wavelengths. The optical emitters102-1 to 102-NN may exhibit continuous wave (CW) operation,quasi-continuous wave (QCW) operation, or pulsed operation.

Referring back to FIG. 1 , in various embodiments, the ToF system 100may comprise an optical source driver 112. The operation of the opticalemitters 102-1 to 102-NN of the optical source 102 may be controlled bythe optical source driver 112, which is configured to generate a drivecurrent 114 that is capable of activating the array of optical emitters102-1 to 102-NN, thereby causing the optical emitters 102-1 to 102-NN toemit photons.

In various embodiments, the array of optical emitters 102-1 to 102-NNmay be an addressable array of optical emitters 102-1 to 102-NN. Thearray of optical emitters 102-1 to 102-NN may be individuallyaddressable where an optical emitter 102-i (shown in FIG. 2A) of thearray of optical emitters 102-1 to 102-NN is addressable independentlyof another optical emitter 102-j of the array of optical emitters 102-1to 102-NN. The drive current 114 provided by the optical source driver112 to the optical source 102 may cause an optical emitter 102-i to beactivated (and thereby emit a photon), while simultaneously causingoptical emitter 102-j to be deactivated (and thereby not emit a photon).In various embodiments, the optical emitters 102-1 to 102-NN may beaddressable as a group or cluster, where one group 108 of opticalemitters 102-1 to 102-NN is addressable independently of another group110 of optical emitters 102-1 to 102-NN.

In various embodiments, the drive current 114 provided by the opticalsource driver 112 to the optical source 102 may cause the group 108 ofoptical emitters 102-1 to 102-NN to be activated (and thereby emit aphoton), while simultaneously causing another group 110 of opticalemitters 102-1 to 102-NN to be deactivated (and thereby not emit aphoton).

Radiation (light) emanating from the optical source 102, collectivelyshown in FIG. 1 as reference numeral 116 using solid arrows, may beincident upon the object 101. The incident radiation 116 is reflectedoff the object 101 to produce reflected radiation 118. It is noted thatalthough incident radiation 116 and reflected radiation 118 arerepresented in FIG. 1 by few arrows, all radiation incident on andreflected from the object 101 may be combined in one beam or cone ofradiation. While some part of the incident radiation 116 may bescattered depending upon the surface features of the object 101, asignificant part of the incident radiation 116 may be reflected, therebyproducing the reflected radiation 118.

The optical receiver 104 receives the reflected radiation 118 andgenerates an output signal 120 in response to the reflected radiation118 striking the optical receiver 104.

FIG. 2B shows an enlarged view of the optical receiver 104, inaccordance with an embodiment.

As depicted in FIG. 2B, the optical receiver 104 may comprise aplurality of radiation-sensitive pixels 104-1 to 104-KK. Although theexample of FIG. 2B illustrates the radiation-sensitive pixels 104-1 to104-KK as being arranged in a square K×K array, other array shapes (e.g.ellipsoidal arrays or circular-shaped arrays) may be possible in otherembodiments.

The radiation-sensitive pixels 104-1 to 104-KK may comprisesingle-photon avalanche diodes (SPADs), photo diodes (PDs), avalanchephoto diodes (APDs), or combinations thereof. In various embodiments,some or all of the plurality of radiation-sensitive pixels 104-1 to104-KK may comprise a plurality of individual light-detecting sensors.The radiation-sensitive pixels 104-1 to 104-KK may generate an eventsignal each time they are hit by a photon.

As shown in FIG. 1 , the ToF system 100 further comprises a processor126 configured to receive the output signal 120 from the opticalreceiver 104 to communicate the signals generated from theradiation-sensitive pixels 104-1 to 104-KK detect photons. The processor126 may analyze the data and determine the proximity of the object 101,or objects, detected by the optical receiver 104 to the ToF system 100based on the output signal 120.

The optical source driver 112 may be programmed to drive all the opticalemitters 102-1 to 102-NN in the array of optical emitters 102-1 to102-NN to generate incident radiation pulses. The optical emitters maybe driven independently, in groups, or a whole. The optical sourcedriver 112 may receive a control signal 134 from the processor 126 thatinitiates the optical source driver 112.

The radiation from the optical source 102 may be projected in theenvironment and reflected from object 101. In various embodiments, theemission may occur in a predetermined timing sequence or atpredetermined timing intervals. The object 101 may reflect the incidentradiation 116 and the arrival times of the pulses of reflected radiation118 at the optical receiver 104 are proportional to twice the distancebetween the object 101 and the ToF system 100, based on the speed oflight in the measurement medium or environment. The arrival time of aphoton may be used to calculate the distance of an object that reflectedthe photon.

The optical source 102 may comprise semiconductor lasers (e.g. VCSELs),while the optical receiver 104 may comprise high speed photodetectors(e.g. SPADs). The optical receiver 104 may be configured to record atleast one of arrival times, pulse shapes, or intensities of the pulsesof reflected radiation 118. Reflected radiation 118 may arrive atdifferent times at the optical receiver 104, depending on the respectivedistances between the different pails of the object 101 or other objectsin the three-dimensional environment and the ToF system 100. Thereflected radiation 118 may be detected synchronously with a timingsignal that is configured to cause the optical source driver 112 togenerate incident radiation 116. In various embodiments, the ToF systemcomprises a clock that may be used controlling optical pulses and may beused for counting time delay between emission of the optical pulse anddetection. The processor 126 may analyze the ToF between emission ofincident radiation 116 travelling towards the object 101 and arrival ofreflected radiation 118 received at the optical receiver 104 todetermine the proximity of the object 101 or objects in thethree-dimensional environment. A plurality of proximity measurements maybe used to generate a comprehensive set of data to accurately determinethe depth (e.g. along the z-axis shown in FIG. 1 ) of the object orobjects in the three-dimensional environment.

In various embodiments, photons counted at a radiation-sensitive pixel,or a group of radiation-sensitive pixels may be categorized based on ToFanalysis to generate a histogram of estimated distances of the object orsurface that reflected the radiation to the radiation-sensitive pixel.The The ToF of a photon sensed at a radiation-sensitive pixel maygrouped in to a bin based on the time of arrival of the photon at theoptical receive 104. The time of arrival, will correlate to distance ofthe object from the ToF system 100. As additional photons are sensedduring a measurement (or exposure), they may also be assigned to a bin.The various bins may accumulate a photon count and the distribution ofphotons in the various bins may be used to estimate the distance fromthe ToF system 100 of the reflective surface measured at theradiation-sensitive pixel.

FIG. 3 depicts a histogram 300 of a photon count.

The vertical axis of the histogram 300 represents the magnitude ofphotons counted. The horizontal axis of the histogram represents time.An optical pulse may be emitted at an initial time 302 by the opticalsource 102. Photons detected during a counting period 306 may then bedetected as light projected by the optical pulse reflects from objectsin the environment and to the optical receiver 104. The photons detectedby the optical receiver may grouped depending on the time between theemission of an optical pulse and the time that they are detected. Thesegroupings may be referred to as bins 304. For example, a 64-nanosecondcounting period 306 may be divided into 64 periods, or bins, each periodcomprising a one nanosecond period. Each bin may also correspond to adistance range. Photons are grouped into the bins 304 that correspond tothe time when they were detected. The detection events are counted togenerates a histogram that reflects the magnitude of photons detectedduring different time periods. These time periods, as will beappreciated, may be correlated to distances based on the relationshipbetween the time of flight of the photons and the speed of light. Itshould be appreciated, that the bins of the histogram may bealternatively represented as distance rather than time because of thisrelationship. As will be appreciated a histogram may be aggregated overmultiple samples.

In various embodiments, a photon count may detect objects in theenvironment. These may be reflected by peaks in the photon count of thehistogram 300. For example, a first peak 308 may correspond to a firstobject located at a first distance from the ToF system 100 in theenvironment, and a second peak 310 may correspond to a second objectlocated at a second distance from the ToF system 100. It will beappreciated by one skilled in the art that ambient light detected by anoptical receiver may result in noise in the signal.

Returning to FIG. 2B, in various embodiments, the radiation-sensitivepixels 104-1 to 104-KK may be divided into sections. A first sections107 may comprise four radiation-sensitive pixels. Eachradiation-sensitive pixel of the optical receiver 104 may belong to asection. Additional sections, (not shown) may comprise differentradiation-sensitive pixels. As will be appreciate, in variousembodiments, the number of radiation-sensitive pixels per section, orthe number of sections per an optical receiver 104 may vary. Forexample, in various embodiments, each section of an optical receiver 104may comprise 16 radiation-sensitive pixels. In various embodiments, eachsection of an optical receiver 104 may comprise 8, 32, 64, or any numberof radiation-sensitive pixels. In various embodiments, a photon countmay generate histogram for each section of radiation-sensitive pixels.For example, if optical receiver 104 comprises 64 groups ofradiation-sensitive pixels, 64 histograms may be generated to range anenvironment for object.

It may be desirable to utilize sections of radiation-sensitive pixels todetect more reflected radiation and improve the signal to noise ratio ofthe output signal 120. In various embodiments, the individualradiation-sensitive pixels of one section may be combined by an OR treefor photon counting.

In various embodiments, the optical receiver 104 may comprise a lens 124to direct photons from the environment into the sections of an opticalreceiver 104.

In various embodiments, the ToF system 100 may comprise one or more timeto digital converters. The one or more TDCs may measure the intervalbetween the emission of incident radiation 116 from the optical source102 and the arrival of reflected radiation 118 at the optical receiver104 and provide it to the processor 126.

Data produced by a ToF system 100 may be used for to determine where tofocus a lens of a camera for an autofocus algorithm. For example, a lensmay be focused at a distance that corresponds to the distance of anobject from a ToF system 100 where the distance of the object from theToF system is determined from an analysis of one or more photon-counthistograms. The ToF system 100 may output a distance at an output 113that may be used for autofocusing.

FIG. 4 depicts a ToF system coupled with a camera in accordance with anembodiment.

The output 113 of the a ToF system 100 may communicate a distance to acamera 402. The camera 402 may use the distance communicated by the ToFsystem to autofocus a lens 404 of the camera according to the distanceprovided to the camera 402 by the ToF system 100. In variousembodiments, the camera 402 and the ToF system may be incorporated intoa single device, like a mobile phone. In various embodiments, the camera402 may comprise the ToF system 100.

FIG. 5 depicts a field of view 500 divided in zones 502.

The zones 502 of the field of view 500, in various embodiments, maycorrespond to the sections of an optical receiver 104. Each zone 502 ofthe field of view may be associated with a corresponding photon-counthistogram produced from a section of the optical receiver that may beused to identify objects within that section of the field of view 500.For example, an optical receive 104 may comprise 64 sections thatproduce 64 photon-count histograms per sample, each photon-counthistogram corresponding to one of 64 zones 502 of the field of view 500.The photon-counts for the sections of the optical receiver 104 may bereferred to as the photon counts of the zones. And, objects in the fieldof view of a zone may be said to be captured when they may be identifiedby the corresponding photon count. The number of zones 502 of the fieldof view 500 may vary from embodiment to embodiment. The field of view500 may be divided in a grid (liked depicted in FIG. 4 ), however, invarious embodiments the field of view 500 may be divided in differentways.

After the ToF system 100 has sampled an environment (generated photoncounts by projecting and receiving light), the ToF system 100 mayanalyze the photon-count histograms to determine a distance of an objectthat is outputted for autofocusing. In various embodiments, thephoton-count histograms generated by a sample may identify multipleobjects in an environment. So, it may advantageous to identify a one ofthe objects as a focal object.

In various embodiment, it may be advantageous to identify a focal sector504 of the field of view 500 where the focal object may be located. Theprocessor 126 may search for a focal object in photon-count histograms(searching for peaks) that correspond to the zones 502 falling withinthe focal sector 504. For example, if the focal sector 504 comprisesfour zones, the processor 126 may analyze the photon-count histogramscorresponding to those four zones to identify a focal object orprospective focal objects.

In various embodiments, the focal sector 504 may comprise more or lesszones 502. For example, the focal sector 504 may comprise 1, 2, 3, 4, 816, or any number of zones in various embodiments. In variousembodiments, the location of the focal sector 504 may be identifiedbased on user input. For example, a user may input a location in thefield of view 500 to be searched for a focal object. This may beaccomplished simply by touching an area in a field of view in a userinterface.

In various embodiments, the focal sector 504 may be identified based ona predetermined parameter. For example, the focal sector 504 may bepredetermined at a central location within a field of view 500. Invarious embodiments, a default setting may locate a focal sector 504 ata predetermined location that may be overridden by user input.

The processor 126 of the ToF system 100 may analyze the photon-countsfor the zones of the focal sector 504 to search for objects. If oneobject is located in the focal sector 504, the processor may output thedistance and that distance may be used for autofocusing. However,complications may arise when more than one object is identified withinthe focal sector 504. This can lead to confusion about the correct focaldistance. And a lens may undesirably shift the focal point of a lensbetween two or more distances thereby causing flicker. Confusion canalso cause a lens to be focused at the wrong distance leading todifficulty capturing an image as desired by a user.

FIG. 6 depicts a Field of view 500 with multiple objects identified in afocal sector 504.

Analyzing each zone of the plurality of zones within the focal sector504, a first prospective focal object 602 located at a first distancefrom the ToF system 100 may be identified in the focal sector 504 usingthe photon counts generated from a ToF sample. Additionally, analyzingeach zone of the plurality of zones within the focal sector 504, asecond prospective focal object 604 located at a second distance fromthe ToF system 100 may also be identified in the focal sector 504. Itmay be desirable to select one of the prospective focal objects as afinal focal object where the lens of a camera may be focused. The firstprospective focal object 602 may be detected in multiple zones of thefocal sector 504. Likewise the second prospective focal object 604 maybe detected in multiple zones of the focal sector 504. The firstprospective focal object 602 and the second prospective focal object 604may also be identified in some overlapping zones. For example, the firstprospective focal object 602 may be identified in all the zones of thefocal sector 504. And, the second prospective focal object 604 may alsobe identified in all the zones of the focal sector 504. Having multipleobjects may make it difficult to determine the appropriate distance foroutput to an autofocus system, or other types of system. Additionalprospective focal objects may also be identified within the focal sector504.

FIG. 7 . depicts zones of a field of view where a first prospectivefocal object and a second prospective focal object are present.

In various embodiments, the field of view 500 may be divided into aplurality of zones. The plurality of zones 502 may correspond tosections of an optical receiver 104 and each zone of the field of viewmay be sampled by a ToF system 100 to generate a photon count for eachzone of the plurality of zones. The example shown in FIG. 7 depicts afield of view 500 divided into 64 zones. It should be appreciated,however, that the field of view 500 may be divided into more or lesszones in various embodiments.

A focal sector 504 (shown in FIG. 6 ) of the field of view 500 may beidentified and each zone of the within the focal sector 504 may beanalyzed to identify a first prospective focal object 602 and toidentify a second prospective focal object 604 present in the focalsector 504. In various embodiments, additional prospective objects maybe identified, and every object identified within any of the zoneswithin the focal sector 504 may be identified as a prospective focalobject. Identifying an object within a zone may be accomplished bylooking for photon-count peaks in the photon count for the correspondingzone and calculating the corresponding distance. Each photon peak ineach zone within the focal sector 504 may be identified and used as adistance for a prospective object.

It may be desirable to select a final focal object from the prospectivefocal objects. In various embodiments, the largest prospective focalobject in the field of view 500 may be selected as the final focalobject. For example, if the first prospective focal object 602 is biggerthan the second prospective focal object 604, it may be advantageous toselect the first prospective focal object as the final focal object. Inthis example, the final-focal-object distance may then comprise thedistance of the first prospective focal object 602 from the ToF system100. In various embodiments, the final-focal-object distance may then beoutputted from the ToF system. It should be appreciated, in variousembodiments the final-focal-object distance may be processed inadditional ways before being output. In various embodiments, thefinal-focal-object distance may be compared with results from previousranges before data is output at least as described with reference toFIG. 8 . If the first prospective focal object 602 is smaller than thesecond prospective focal object 604, second prospective focal object 604may be selected as the final-focal-object. In various embodiments, thesmall object may be selected as the final focal object instead of thelarger object.

It should also be appreciated that additional objects may be detected inthe focal sector 504 so there may be more than two prospective focalobjects. In various embodiments, each object detected within the focalsector 504 will be considered a prospective focal object.

Once objects within the focal sector 504 have been identified, it may bedesirable to determine the sizes of the objects identified within thefocal sector 504 to determine which object should be selected as a finalfocal object. This may be accomplished by counting the number of zoneswhere the respective prospective focal objects may be identified in thehistogram for the corresponding zone. To find all the zones where aprospective focal object is present, the processor may analyze thephoton counts of all the zones of the field of view. For example, if thefirst prospective focal object 602 is identified as being at a distanceof 300 mm, the processor may analyze the photon counts of every zone ofthe field of view 500 to find every zone with a photon count that has apeak at 300 mm plus and minus the threshold value. The first prospectivefocal object 602 may be considered to be present in every zone that hasa photon count peak within the threshold distance of the firstprospective focal object 602. If the threshold value is 5% of thedistance (15 mm), the processor 126 will count every zone with a photoncount that has a peak between 285 mm and 315 mm as a zone as being azone where the first prospective focal object 602 is present. If thesecond prospective focal object 604 is determined to be located at 1500mm, using the same 5% threshold, the processor 126 will count every zonewith a photon count that has a peak between 1425 mm and 1575 mm as azone where the second prospective focal object 604 is located. Invarious embodiments, the threshold may comprise a set distance ratherthan a percentage. The processor 126 may also store data related to thephoton counts (such as the number of zones of each prospective focalobject) in a memory 132.

In various embodiments, the prospective focal object that is present inthe most zones of the field of view 500 may be considered the largestobject and selected as the final-focal object. In FIG. 7 , a firstpattern is used to show the zones where the first prospective focalobject 602 is present, a second pattern is used to show the zones wherethe second prospective focal object 604 is present, and a third patternis used to show where both the first prospective focal object 602 andthe second prospective focal object 604 are present. These zones may becounted to reveal that the first prospective focal object 602 is presentin 25 zones. The second prospective focal object 604 is present in 39zones. In this example, the second prospective focal object 604 islarger than the first prospective focal object 602 so it may be selectedas the final focal object. In various embodiments, a user may be givenan option to select either the larger object (the second prospectivefocal object 604 in this example) or the smaller object (the firstprospective focal object 602 in this example).

In various embodiments, the processor 126 may search the photon countfor each zone of the field of view 500 one at a time searching for peakswithin a threshold of the first prospective focal object 602. Forexample, the processor may identify the peaks in a histogram for a firstzone. If any of those peaks occur within a threshold distance of thedistance of the first prospective focal object 602, then that zone maybe identified as a zone where the first prospective focal object 602 ispresent. Then, the processor 126 may search the photon count for eachzone of the field of view 500 one at a time searching for peaks locatedwithin a threshold of the second prospective focal object 604. This maycontinue for each prospective object identified with the focal sector504.

In various embodiments, the processor 126 may search a first zone of theplurality of zone for a peak located within the threshold of the firstprospective focal object 602. Then, the processor 126 may search thefirst zone for a peak located within the threshold of the secondprospective focal object 604. Searches in the first zone may continuefor each object identified with the focal sector 504. Then, this can berepeated for every other zone of the field of view 500.

As will be appreciated, due to variations in the photon count, thecalculated distance of an object may vary from zone to zone. Forexample, the first prospective focal object 602 may be identified at adistance of 302 mm in a first zone of the focal sector 504 while thesame object may be identified at 298 mm in a second zone of the focalsector 504 It may be undesirable to search each zone for peaks locatedwithin a threshold distance of 302 mm and perform another search forpeaks located within the threshold distance of 298 mm because they maybe likely to produce the same or similar results, which may beinefficient. To avoid redundant searches, it may be desirable to compareprospective focal objects to each other to determine if they are in athreshold distance of each other and discard prospective objects thatare within the threshold of another prospective object.

Returning to FIG. 7 , the first prospective focal object 602 appears inall four zones of the focal sector 504. The photon counts for each ofthe four zones may yield slightly different distances for the firstprospective focal object 602. Likewise, the second prospective focalobject 604 appears in 3 of the 4 zones of the focal sector 504. Thephoton counts for these three zones may yield three slightly differentdistances. Searching each zone for peaks located within a thresholddistance of the four slightly different distances associated with thefirst prospective object 602, and searching the histograms of each zonefor peaks located within a threshold distance of the three slightlydifferent distances associated with the second prospective object 604perform may yield very similar results an be inefficient. It may bepreferable to perform one search for the first prospective object 602and one search for he second prospective object 604.To avoidredundancies, prospective objects within a threshold of anearlier-identified prospective object may be removed from considerationso that only one search of the zones is performed for each prospectiveobject. This may be accomplished by storing potential values in a memory132 and comparing potential values with each other before performingsearches in the zones of the field of view 500.

In various embodiments, the distances of prospective focal objectswithin a threshold of each other may be averaged before counting thezones where the prospective objects are present. For example, the fourdistances from the first prospective focal object 602 within the focalsector 504 may be averaged if they are within a threshold of each other.Then, the histograms of each zone may be searched for peaks located withthe threshold distance of the averaged value. And, the same may be donefor the three distances from the second prospective focal object 604within the focal sector 504.

Once a final focal object has been selected, in various embodiments, itmay be advantageous to average the distances from each of the zoneswhere the selected object is present. For example, in FIG. 7 , thesecond prospective focal object 604 is identified in 37 zones. Thedistance of the second prospective focal object 604 detected in eachzone may be averaged together to find an averaged focal distance. Thisvalue may be output by the ToF system 100. This may be desirable toreduce the standard deviation of the value outputted by the ToF system100. In various embodiments, it may also be desirable to average thesignal strength, ambient light (noise), or the standard deviation forthe final object.

In various embodiments, temporal filtering may also be utilized beforethe outputting the results. Temporal filtering may aid to reduceautofocus flicker caused by uneven changes in the distance output by theToF system 100. For example, after a first ranging, a ToF system mayoutput a first distance, and after a second ranging the ToF system mayoutput a second distance. This may occur when the size of theprospective objects are close to each other. Output from subsequentranges may also switch between the two distances. When paired with anautofocus algorithm and device, this may cause the autofocus to switch,undesirably, from the first distance to the second distance and back.This may detract from a user's experience. Temporal filtering may smoothuneven switching of the output of the ToF system.

In various embodiments, temporal filtering may comprise storing thedistance previously output by the ToF system and storing distancesselected from previous ranges as a final-focal-object distance (that mayor may not have been output). In various embodiments, this informationmay be stored in the memory 132 and accessed by the processor 126 beforedata is output at the output 113. The distance selected from a currentranging (the distance of the final focal object or averaged distance ofall the zones where the final focal object is present in variousembodiments), may be compared with distances from previous ranges anddistances previously output the ToF system 100.

FIG. 8 depicts a flowchart of a method for outputting a distance withtemporal filtering in accordance with an embodiment.

In various embodiments, at a step 800, the distances for prospectivefocal objects are identified during a current ranging. In variousembodiments, this may be accomplished by searching the focal sector 504as described elsewhere in this disclosure. A final focal object may beselected from the prospective focal objects as disclosed herein. Thismay comprise selecting the focal object that is identified in more zonesof a field of view 500.

At a step 802, the distances of the prospective focal objects of thecurrent ranging are compared with the last distance outputted by the ToFsystem 100. If the last distance outputted is found among the distancesof the prospective focal objects (of the current ranging), at step 804,the final-focal-object distance (of the current ranging) may be comparedwith the last distance outputted. If the final-focal-object distance (ofthe current ranging) is equal to the last distance output, thefinal-final-focal object distance (of the current ranging) may beoutputted at step 806. If they are not equal, at a step 808, thefinal-focal-object distance may compared with the final-focal-objectdistance of the previous five iterations (ranges) of the method toselect a focal object. If the final-focal-object distance of the presentiteration is equal to the previous five final focal distances, thefinal-focal-object distance of the present iteration may be output at810. Otherwise, the last distance outputted by the ToF system 100 may beoutputted again at step 812.

After step 802, if the last distance outputted (from the most recentiteration) is not found among the distances of the prospective focalobjects, the final-focal-object distance of the present iteration may becompared with the final-focal-object distance of the previous two rangesat step 814. If the final-focal-object distance of the present iterationis equal to the final-focal-object distance of the previous two ranges,the final focal object distance (of the current iteration) may be outputat step 816. Otherwise, the last distance outputted by the ToF system100 may be output again at step 818.

In various embodiments, the number of ranges used for comparisons mayvary. For example, at a step 808, the final-focal-object distance of apresent range may be compared with the distance of the final focalobject from 3, 6, 8, 10 or any number of previous ranges. In variousembodiments the final-focal-object distance may not need to be equal toevery previous range. For example, the final-focal-object distance mayonly need to be equal to 4 of 5 previous ranges. Similarly, at a step814, the final focal distance of a present range may be compared withthe distance of the final focal object from more than two previousranges or less than two previous ranges. In various embodiments,parameters may be adapted for different applications or design criteria.

Various embodiments of the temporal filter will now be further explainedusing some examples.

TABLE 1 Prospective Focal Objects Found: 300 mm-38 zones 1.8 m-36 zonesLast Distance Reported: 1.8 m Distances From Previous Ranges: 300 mm 1.8m 300 mm 1.8 m 300 mm Output Distance: 1.8 m

TABLE 2 Prospective Objects Found: 300 mm-38 zones 1.8 m-37 zones LastDistance Reported: 1.8 m Previous Ranges: 300 mm 300 mm 300 mm 300 mm300 mm Output Distance: 300 mm

Using the data from Table 1 as an example, in various embodiments, 300mm may be selected as final-focal-object distance because it appears inmore zones than the alternative. For the example, the last distanceoutput is 1.8M, which is found among the prospective focal objects, sothe final-focal-object distance (300 mm) may be compared with the lastdistance output at 804. From table 1, the last distance output (1.8M)does not equal the final-focal-object distance (300 mm) so, at step 808,the final-focal-object distance is compared with the distances from thelast 5 ranges. In this example, the final-focal-object distance is notequal to the last 5 ranges so the last distance output (1.8M) may beoutputted again.

Using the data from Table 2 for an example, in various embodiments, 300mm may, again, be selected as final-focal-object distance because itappears in more zones than the alternative. The last distance output is1.8M, which is found among the prospective focal objects so thefinal-focal-object distance (300 mm) may be compared with the lastdistance output at step 804. From table 2, the last distance output(1.8M) does not equal the final-focal-object distance (300 mm) so thefinal-focal-object distance is compared with the distances from the last

TABLE 3 Prospective Objects Found: 300 mm-40 Zones 600 mm-20 zones LastDistance Reported: 1.8 m Previous Ranges: 1.8 m 1.8 m 300 mm 300 mm 1.8m Output Distance: 1.8 m5 ranges. In this example, the final-focal-object distance is equal tothe last 5 ranges so the final-focal-object distance (300 mm) may beoutputted.

Using data from Table 3 for another example, in various embodiments, 300mm may, again, be selected as final-focal-object distance because itappears in more zones than the alternative. The last distance output is1.8M, which is not found among the prospective focal objects so thefinal-focal-object distance (300 mm) may be compared with the previoustwo

TABLE 4 Prospective Objects Found: 300 mm-40 Zones 600 mm-20 zones LastDistance Reported: 1.8 m Previous Ranges: 300 mm 1.8 m 300 mm 300 mm 1.8m Output Distance: 1.8 mranges at 914. In this example, the final focal object distance (300 mm)is not equal to the last two ranges so the last distance reported (1.8M)may be output again at 818.

Using data from Table 4 for another example, in various embodiments, 300mm may, again, be selected as final-focal-object distance because itappears in more zones than the alternative. The last distance output is1.8M, which is not found among the prospective focal objects so thefinal-focal-object distance (300 mm) may be compared with the previoustwo ranges at 914. In this example, the final-focal-object distance (300mm) is equal to the last two ranges so the final-focal-object distance(300 mm) may be output at 816.

FIG. 9 depicts a method 900 to identify a focal object in a field ofview in accordance with an embodiment.

In various embodiments, the method 900 may comprise at a step 902,dividing a field of view into a plurality of zones; at a step 904,identifying a focal sector occupying a subset of the plurality of zones;at a step 906, analyzing each zone of the plurality of zones within thefocal sector to identify a first prospective focal object and toidentify a second prospective focal object; at a step 908, determining asize of the first prospective focal object in the field of view anddetermining a size of the second prospective focal object in the fieldof view; and at a step 910 selecting a final focal object from the firstprospective focal object and the second prospective focal object bycomparing the size of the first prospective focal object with the sizeof the second prospective focal object; and at a step 912, focusing alens of a camera depending on the final focal object.

In various embodiments, the method 900 may comprise, wherein the focalsector is identified based on user input.

In various embodiments, the method 900 may comprise, wherein the focalsector is identified based on a predetermined parameter.

In various embodiments, the method 900 may comprise, wherein the size ofthe first prospective focal object is determined by analyzing each zoneof the plurality of zones and counting a first number of zones where thefirst prospective focal object is present and wherein the size of thesecond prospective focal object is determined by analyzing each zone ofthe plurality of zones and counting a second number of zones where thesecond prospective focal object is present.

In various embodiments, the method 900 may further comprise sampling thefield of view with a Time of Flight system to generate a photon countfor each zone of the plurality of zones and wherein the final focalobject is located at a final-focal-object distance from the Time ofFlight system.

In various embodiments, the method 900 may comprise, wherein countingthe first number of zones comprises analyzing the photon count for eachzone of the plurality of zones to identify each zone of the plurality ofzones that captures an object within a threshold distance of the firstprospective focal object and wherein counting the second number of zonescomprises analyzing the photon count for each zone of the plurality ofzones to identify each zone of the plurality of zones that captures anobject within the threshold distance of the second prospective focalobject.

In various embodiments, the method 900 may further comprise averagingthe final-focal-object distance with a distance of each object within athreshold distance of the final focal object in each zone of theplurality of zones where the final focal object is present.

In various embodiments, the method 900 may comprise wherein, the fieldof view comprises a field of view of the Time of Flight System.

In various embodiments, the method 900 may further comprise determiningthat the final-focal-object distance is equal to a last distancepreviously outputted by the Time of Flight system and outputting thefinal-focal-object distance from the Time of Flight system to focus thelens of the camera.

In various embodiments, the method 900 may comprise determining thatthat the final-focal-object distance does not equal a last distancepreviously outputted by the Time of Flight system; determining that thefinal-focal-object distance is equal to a plurality ofpreviously-identified final-focal-object distances; and outputting, bythe Time of Flight system, the final-focal-object distance to focus thelens of the camera.

In various embodiments, the method 900 may further comprise determiningthat that the final-focal-object distance does not equal a last distancepreviously outputted by the Time of Flight system; determining that thefinal-focal-object distance does not equal to a plurality ofpreviously-identified final-focal-object distances; and outputting, bythe Time of Flight system, the last distance previously outputted by theTime of Flight system to focus the lens of the camera.

In various embodiments, the method 900 may comprise, determining that alast distance previously outputted by the Time of Flight system is notequal to a distance of the first prospective focal object; determiningthat the last distance previously outputted by the Time of Flight systemis not equal to a distance of the second prospective focal object;determining that the final-focal-object distance does not equal aplurality of previously-identified final-focal-object distances; andoutputting, by the Time of Flight system the last distance previouslyoutputted by the Time of Flight system to focus the lens of the camera.

In various embodiments, the method 900 may further comprise determiningthat a last distance previously outputted by the Time of Flight systemis not equal to a distance of the first prospective focal object;determining that the last distance previously outputted by the Time ofFlight system is not equal to a distance of the second prospective focalobject; determining that the final-focal-object distance is equal to aplurality of previously-identified final-focal-object distances; andoutputting, by the Time of Flight system, the final-focal-objectdistance to focus the lens of the camera.

In various embodiments, the method 900 may further comprise analyzingeach zone of the plurality of zones within the focal sector to identifya third prospective focal object; determining a size of the thirdprospective focal object in the field of view; and wherein the size ofthe third prospective focal object is also compared with the size of thefirst prospective focal object and the second prospective focal objectto select the final focal object from the first prospective focalobject, the second prospective focal object, and the third prospectivefocal object.

FIGS. 10A and 10B depict a method to identify a focal object in a fieldof view in accordance with an embodiment.

In various embodiments, the method 1000 may comprising, a step 1002dividing a field of view of a Time of Flight system into a plurality ofzones; at a step 1004, sampling the field of view with the Time ofFlight system to generate a photon count for each zone of the pluralityof zones; at a step 1006, identifying a focal sector occupying a subsetof the plurality of zones; at a step 1008 analyzing each zone of theplurality of zones within the focal sector to identify a firstprospective focal object and identify a second prospective focal object;at a step 1010 determining a size of the first prospective focal objectby analyzing a photon count for each zone of the plurality of zones tocount each zone of the plurality of zones that captures an object withina threshold distance of the first prospective focal object; at a step1012, determining a size of the second prospective focal object byanalyzing the photon count for each zone of the plurality of zones tocount each zone of the plurality of zones that captures an object withinthe threshold distance of the second prospective focal object; at a step1014, selecting a final focal object from the first prospective focalobject and the second prospective focal object by comparing the size ofthe first prospective focal object with the size of the secondprospective focal object, the final focal object being located at afinal-focal-object distance from the Time of Flight system; and at astep 1016, focusing a lens of a camera depending on thefinal-focal-object distance.

In various embodiments the method 1000 may further comprise, adjustingthe final-focal-object distance by averaging the final-focal-objectdistance with a distance of each object within the threshold distance ofthe final focal object in each zone of the plurality of zones where thefinal focal object is present.

In various embodiments the method 1000 may further comprise analyzingeach zone of the plurality of zones within the focal sector to identifya third prospective focal object; determining a size of the thirdprospective focal object by analyzing the photon count for each zone ofthe plurality of zones to count each zone of the plurality of zones thatcaptures an object within the threshold distance of the thirdprospective focal object; and wherein the size of the third prospectivefocal object is compared with the size of the first prospective focalobject and the size of the second prospective focal object to select thefinal focal object.

In various embodiments the method 1000 may further comprise determiningthat the final-focal-object distance is equal to a last distancepreviously outputted by the Time of Flight system and outputting thefinal-focal-object distance from the Time of Flight system to focus thelens of the camera.

In various embodiments the method 1000 may further comprise determiningthat that the final-focal-object distance does not equal a last distancepreviously outputted by the Time of Flight system; determining that thefinal-focal-object distance is equal to a plurality ofpreviously-identified final-focal-object distances; and outputting, bythe Time of Flight system, the final-focal-object distance to focus thelens of the camera.

EXAMPLE 1

A method to identify a focal object in a field of view, the methodincluding: dividing a field of view into a plurality of zones;identifying a focal sector occupying a subset of the plurality of zones;analyzing each zone of the plurality of zones within the focal sector toidentify a first prospective focal object and to identify a secondprospective focal object; determining a size of the first prospectivefocal object in the field of view and determining a size of the secondprospective focal object in the field of view; selecting a final focalobject from the first prospective focal object and the secondprospective focal object by comparing the size of the first prospectivefocal object with the size of the second prospective focal object; andfocusing a lens of a camera depending on the final focal object.

EXAMPLE 2

The method of Example 1, further wherein, the focal sector is identifiedbased on user input.

EXAMPLE 3

The method of Example 1 or Example 2, wherein the focal sector isidentified based on a predetermined parameter.

EXAMPLE 4

The method of Example 1 to Example 3, wherein the size of the firstprospective focal object is determined by analyzing each zone of theplurality of zones and counting a first number of zones where the firstprospective focal object is present and wherein the size of the secondprospective focal object is determined by analyzing each zone of theplurality of zones and counting a second number of zones where thesecond prospective focal object is present.

EXAMPLE 5

The method of Example 1 to Example 4, further including sampling thefield of view with a Time of Flight system to generate a photon countfor each zone of the plurality of zones and wherein the final focalobject is located at a final-focal-object distance from the Time ofFlight system.

EXAMPLE 6

The method of Example 1 to Example 5, wherein counting the first numberof zones comprises analyzing the photon count for each zone of theplurality of zones to identify each zone of the plurality of zones thatcaptures an object within a threshold distance of the first prospectivefocal object and wherein counting the second number of zones comprisesanalyzing the photon count for each zone of the plurality of zones toidentify each zone of the plurality of zones that captures an objectwithin the threshold distance of the second prospective focal object.

EXAMPLE 7

The method of Example 1 to Example 6, further including averaging thefinal-focal-object distance with a distance of each object within athreshold distance of the final focal object in each zone of theplurality of zones where the final focal object is present.

EXAMPLE 8

The method of Example 1 to Example 7, further wherein, the field of viewcomprises a field of view of the Time of Flight System.

EXAMPLE 9

The method of Example 1 to Example 8, further including: determiningthat the final-focal-object distance is equal to a last distancepreviously outputted by the Time of Flight system and outputting thefinal-focal-object distance from the Time of Flight system to focus thelens of the camera.

EXAMPLE 10

The method of Example 1 to Example 9, further including: determiningthat that the final-focal-object distance does not equal a last distancepreviously outputted by the Time of Flight system; determining that thefinal-focal-object distance is equal to a plurality ofpreviously-identified final-focal-object distances; and outputting, bythe Time of Flight system, the final-focal-object distance to focus thelens of the camera.

EXAMPLE 11

The method of Example 1 to Example 10, further including: determiningthat that the final-focal-object distance does not equal a last distancepreviously outputted by the Time of Flight system; determining that thefinal-focal-object distance does not equal to a plurality ofpreviously-identified final-focal-object distances; and outputting, bythe Time of Flight system, the last distance previously outputted by theTime of Flight system to focus the lens of the camera.

EXAMPLE 12

The method of Example 1 to Example 11, further including: determiningthat a last distance previously outputted by the Time of Flight systemis not equal to a distance of the first prospective focal object;determining that the last distance previously outputted by the Time ofFlight system is not equal to a distance of the second prospective focalobject; determining that the final-focal-object distance does not equala plurality of previously-identified final-focal-object distances; andoutputting, by the Time of Flight system the last distance previouslyoutputted by the Time of Flight system to focus the lens of the camera.

EXAMPLE 13

The method of Example 1 to Example 12 further including: determiningthat a last distance previously outputted by the Time of Flight systemis not equal to a distance of the first prospective focal object;determining that the last distance previously outputted by the Time ofFlight system is not equal to a distance of the second prospective focalobject; determining that the final-focal-object distance is equal to aplurality of previously-identified final-focal-object distances; andoutputting, by the Time of Flight system, the final-focal-objectdistance to focus the lens of the camera.

EXAMPLE 14

The method of Example 1 to Example 13, further including analyzing eachzone of the plurality of zones within the focal sector to identify athird prospective focal object; determining a size of the thirdprospective focal object in the field of view; and wherein the size ofthe third prospective focal object is also compared with the size of thefirst prospective focal object and the second prospective focal objectto select the final focal object from the first prospective focalobject, the second prospective focal object, and the third prospectivefocal object.

EXAMPLE 15

A system including: a Time of Flight system configured to detect aphoton count for each zone of a plurality of zones of a field of view; aprocessor in communication with the Time of Flight system and a memorycomprising an instruction set to be executed in the processor, theinstruction set when executed causing the processor to: identify a focalsector occupying a subset of the plurality of zones, analyze each zoneof the plurality of zones within the focal sector to identify a firstprospective focal object and to identify a second prospective focalobject, determine a size of the first prospective focal object anddetermine a size of the second prospective focal object, and select afinal focal object from the first prospective focal object and thesecond prospective focal object by comparing the size of the firstprospective focal object with the size of the second prospective focalobject, the final focal object being located at a final-focal-objectdistance from the Time of Flight system; and a lens focus systemconfigured to focus a lens of a camera depending on the final focalobject.

EXAMPLE 16

The system of Example 15, wherein the Time of Flight system includes: anoptical source configured to emit light into an environment; and anoptical receiver comprising a plurality of light-sensitive pixels todetect photons being reflected from the environment, the plurality oflight-sensitive pixels arranged in a plurality of sections correspondingto the plurality of zones.

EXAMPLE 17

The system of Example 15 or Example 16, wherein the Time of Flightsystem includes the processor and wherein the processor receives thephoton count for each zone of the plurality of zones from the opticalreceiver.

EXAMPLE 18

The system of Example 15 to Example 17, wherein the optical receivercomprises an optical-receiver lens to direct photons to the plurality ofsections.

EXAMPLE 19

A method to identify a focal object in a field of view, the methodincluding: dividing a field of view of a Time of Flight system into aplurality of zones; sampling the field of view with the Time of Flightsystem to generate a photon count for each zone of the plurality ofzones; identifying a focal sector occupying a subset of the plurality ofzones; analyzing each zone of the plurality of zones within the focalsector to identify a first prospective focal object and identify asecond prospective focal object; determining a size of the firstprospective focal object by analyzing a photon count for each zone ofthe plurality of zones to count each zone of the plurality of zones thatcaptures an object within a threshold distance of the first prospectivefocal object; determining a size of the second prospective focal objectby analyzing the photon count for each zone of the plurality of zones tocount each zone of the plurality of zones that captures an object withinthe threshold distance of the second prospective focal object; selectinga final focal object from the first prospective focal object and thesecond prospective focal object by comparing the size of the firstprospective focal object with the size of the second prospective focalobject, the final focal object being located at a final-focal-objectdistance from the Time of Flight system; and focusing a lens of a cameradepending on the final-focal-object distance.

EXAMPLE 20

The method of Example 19, further including adjusting thefinal-focal-object distance by averaging the final-focal-object distancewith a distance of each object within the threshold distance of thefinal focal object in each zone of the plurality of zones where thefinal focal object is present.

EXAMPLE 21

The method of Example 19 or Example 20, further including analyzing eachzone of the plurality of zones within the focal sector to identify athird prospective focal object; determining a size of the thirdprospective focal object by analyzing the photon count for each zone ofthe plurality of zones to count each zone of the plurality of zones thatcaptures an object within the threshold distance of the thirdprospective focal object; and wherein the size of the third prospectivefocal object is compared with the size of the first prospective focalobject and the size of the second prospective focal object to select thefinal focal object.

EXAMPLE 22

The method of Example 19 to Example 21, further including determiningthat the final-focal-object distance is equal to a last distancepreviously outputted by the Time of Flight system and outputting thefinal-focal-object distance from the Time of Flight system to focus thelens of the camera.

While this invention has been described with reference to illustrativeembodiments, this description is not intended to be construed in alimiting sense. Various modifications and combinations of theillustrative embodiments, as well as other embodiments of the invention,will be apparent to persons skilled in the art upon reference to thedescription. It is therefore intended that the appended claims encompassany such modifications or embodiments.

What is claimed is:
 1. A device, comprising: a non-transitory memorystorage comprising instructions; and a processor in communication withthe non-transitory memory storage, wherein the processor is configuredto execute the instructions to: divide a field of view of a Time ofFlight system into a plurality of zones; sample the field of view withthe Time of Flight system to generate a photon count for each zone ofthe plurality of zones; identify a focal sector occupying a subset ofthe plurality of zones; analyze each zone of the plurality of zoneswithin the focal sector to identify a first prospective focal object anda second prospective focal object; determine a size of the firstprospective focal object by analyzing a photon count for each zone ofthe plurality of zones to count each zone of the plurality of zones thatcaptures an object within a threshold distance of the first prospectivefocal object; determine a size of the second prospective focal object byanalyzing the photon count for each zone of the plurality of zones tocount each zone of the plurality of zones that captures an object withinthe threshold distance of the second prospective focal object; select afinal focal object from the first prospective focal object and thesecond prospective focal object by comparing the size of the firstprospective focal object with the size of the second prospective focalobject, the final focal object being located at a final-focal-objectdistance from the Time of Flight system; and focus a lens of a camerabased on the final-focal-object distance.
 2. The device of claim 1,wherein the processor is configured to execute the instructions toadjust the final-focal-object distance by averaging thefinal-focal-object distance with a distance of each object within thethreshold distance of the final focal object in each zone of theplurality of zones where the final focal object is present.
 3. Thedevice of claim 1, wherein the processor is configured to execute theinstructions to: analyze each zone of the plurality of zones within thefocal sector to identify a third prospective focal object; and determinea size of the third prospective focal object by analyzing the photoncount for each zone of the plurality of zones to count each zone of theplurality of zones that captures an object within the threshold distanceof the third prospective focal object, wherein the size of the thirdprospective focal object is compared with the size of the firstprospective focal object and the size of the second prospective focalobject to select the final focal object.
 4. The device of claim 1,wherein the processor is configured to execute the instructions todetermine that the final-focal-object distance is equal to a lastdistance previously outputted by the Time of Flight system andoutputting the final-focal-object distance from the Time of Flightsystem to focus the lens of the camera.
 5. The device of claim 1,wherein the processor is configured to execute the instructions to:determine that that the final-focal-object distance does not equal alast distance previously outputted by the Time of Flight system;determine that the final-focal-object distance is equal to a pluralityof previously-identified final-focal-object distances; and output, bythe Time of Flight system, the final-focal-object distance to focus thelens of the camera.
 6. The device of claim 1, wherein the processor isconfigured to execute the instructions to: determine that that thefinal-focal-object distance does not equal a last distance previouslyoutputted by the Time of Flight system; determine that thefinal-focal-object distance does not equal to a plurality ofpreviously-identified final-focal-object distances; and output, by theTime of Flight system, the last distance previously outputted by theTime of Flight system to focus the lens of the camera.
 7. The device ofclaim 1, wherein the processor is configured to execute the instructionsto: determine that a last distance previously outputted by the Time ofFlight system is not equal to a distance of the first prospective focalobject; determine that the last distance previously outputted by theTime of Flight system is not equal to a distance of the secondprospective focal object; determine that the final-focal-object distancedoes not equal a plurality of previously-identified final-focal-objectdistances; and output, by the Time of Flight system the last distancepreviously outputted by the Time of Flight system to focus the lens ofthe camera.
 8. The device of claim 1, wherein the processor isconfigured to execute the instructions to: determine that a lastdistance previously outputted by the Time of Flight system is not equalto a distance of the first prospective focal object; determine that thelast distance previously outputted by the Time of Flight system is notequal to a distance of the second prospective focal object; determinethat the final-focal-object distance is equal to a plurality ofpreviously-identified final-focal-object distances; and output, by theTime of Flight system, the final-focal-object distance to focus the lensof the camera.
 9. A device, comprising: a non-transitory memory storagecomprising instructions; and a processor in communication with thenon-transitory memory storage, wherein the processor is configured toexecute the instructions to: divide a field of view of a Time of FlightSystem into a plurality of zones; identify a focal sector occupying asubset of the plurality of zones; analyze each zone of the plurality ofzones within the focal sector to identify a first prospective focalobject and a second prospective focal object; select a final focalobject from the first prospective focal object and the secondprospective focal object by comparing a size of the first prospectivefocal object with a size of the second prospective focal object; andfocus a lens of a camera depending on the final focal object.
 10. Thedevice of claim 9, wherein the focal sector is identified based on userinput.
 11. The device of claim 9, wherein the focal sector is identifiedbased on a predetermined parameter.
 12. The device of claim 9, whereinthe processor is configured to execute the instructions to: analyze eachzone of the plurality of zones and counting a first number of zoneswhere the first prospective focal object is present to determine thesize of the first prospective focal object; and analyze each zone of theplurality of zones and counting a second number of zones where thesecond prospective focal object is present to determine the size of thesecond prospective focal object.
 13. The device of claim 12, wherein theprocessor is configured to execute the instructions to sample the fieldof view with a Time of Flight system to generate a photon count for eachzone of the plurality of zones, and wherein the final focal object islocated at a final-focal-object distance from the Time of Flight system.14. The device of claim 13, wherein the processor is configured toexecute the instructions to: analyze the photon count for each zone ofthe plurality of zones to identify each zone of the plurality of zonesthat captures an object within a threshold distance of the firstprospective focal object; and analyze the photon count for each zone ofthe plurality of zones to identify each zone of the plurality of zonesthat captures an object within the threshold distance of the secondprospective focal object.
 15. The device of claim 13, wherein theprocessor is configured to execute the instructions to average thefinal-focal-object distance with a distance of each object within athreshold distance of the final focal object in each zone of theplurality of zones where the final focal object is present.
 16. Anon-transitory computer readable media storing computer instructions,that when executed by a processor, cause the processor to: divide afield of view of a Time of Flight System into a plurality of zones;identify a focal sector occupying a subset of the plurality of zones;analyze each zone of the plurality of zones within the focal sector toidentify a first prospective focal object and a second prospective focalobject; select a final focal object from the first prospective focalobject and the second prospective focal object by comparing a size ofthe first prospective focal object with a size of the second prospectivefocal object; and focus a lens of a camera depending on the final focalobject.
 17. The non-transitory computer readable media of claim 16,wherein the Time of Flight system comprises: an optical sourceconfigured to emit light into an environment; and an optical receivercomprising a plurality of light-sensitive pixels to detect photons beingreflected from the environment, the plurality of light-sensitive pixelsarranged in a plurality of sections corresponding to the plurality ofzones.
 18. The non-transitory computer readable media of claim 17,wherein the optical receiver comprises an optical-receiver lens todirect photons to the plurality of sections.
 19. The non-transitorycomputer readable media of claim 16, wherein the computer instructions,when executed by the processor, cause the processor to determine thatthe final-focal-object distance is equal to a last distance previouslyoutputted by the Time of Flight system and outputting thefinal-focal-object distance from the Time of Flight system to focus thelens of the camera.
 20. The non-transitory computer readable media ofclaim 16, wherein the computer instructions, when executed by theprocessor, cause the processor to: sample the field of view with a Timeof Flight system to generate a photon count for each zone of theplurality of zones, and wherein the final focal object is located at afinal-focal-object distance from the Time of Flight system; and averagethe final-focal-object distance with a distance of each object within athreshold distance of the final focal object in each zone of theplurality of zones where the final focal object is present.